Conditional Random Field Layer Research Articles

BioBBC: a multi-feature model that enhances the detection of biomedical entities

The rapid increase in biomedical publications necessitates efficient systems to automatically handle Biomedical Named Entity Recognition (BioNER) tasks in unstructured text. However, accurately detecting biomedical entities is quite challenging due to the complexity of their names and the frequent use of abbreviations. In this paper, we propose BioBBC, a deep learning (DL) model that utilizes multi-feature embeddings and is constructed based on the BERT-BiLSTM-CRF to address the BioNER task. BioBBC consists of three main layers; an embedding layer, a Long Short-Term Memory (Bi-LSTM) layer, and a Conditional Random Fields (CRF) layer. BioBBC takes sentences from the biomedical domain as input and identifies the biomedical entities mentioned within the text. The embedding layer generates enriched contextual representation vectors of the input by learning the text through four types of embeddings: part-of-speech tags (POS tags) embedding, char-level embedding, BERT embedding, and data-specific embedding. The BiLSTM layer produces additional syntactic and semantic feature representations. Finally, the CRF layer identifies the best possible tag sequence for the input sentence. Our model is well-constructed and well-optimized for detecting different types of biomedical entities. Based on experimental results, our model outperformed state-of-the-art (SOTA) models with significant improvements based on six benchmark BioNER datasets.

Personalized Driver Gene Prediction Using Graph Convolutional Networks with Conditional Random Fields.

Cancer is a complex and evolutionary disease mainly driven by the accumulation of genetic variations in genes. Identifying cancer driver genes is important. However, most related studies have focused on the population level. Cancer is a disease with high heterogeneity. Thus, the discovery of driver genes at the individual level is becoming more valuable but is a great challenge. Although there have been some computational methods proposed to tackle this challenge, few can cover all patient samples well, and there is still room for performance improvement. In this study, to identify individual-level driver genes more efficiently, we propose the PDGCN method. PDGCN integrates multiple types of data features, including mutation, expression, methylation, copy number data, and system-level gene features, along with network structural features extracted using Node2vec in order to construct a sample-gene interaction network. Prediction is performed using a graphical convolutional neural network model with a conditional random field layer, which is able to better combine the network structural features with biological attribute features. Experiments on the ACC (Adrenocortical Cancer) and KICH (Kidney Chromophobe) datasets from TCGA (The Cancer Genome Atlas) demonstrated that the method performs better compared to other similar methods. It can identify not only frequently mutated driver genes, but also rare candidate driver genes and novel biomarker genes. The results of the survival and enrichment analyses of these detected genes demonstrate that the method can identify important driver genes at the individual level.

Application of BiLSTM-CRF model with different embeddings for product name extraction in unstructured Turkish text

AbstractNamed entity recognition (NER) plays a pivotal role in Natural Language Processing by identifying and classifying entities within textual data. While NER methodologies have seen significant advancements, driven by pretrained word embeddings and deep neural networks, the majority of these studies have focused on text with well-defined grammar and structure. A significant research gap exists concerning NER in informal or unstructured text, where traditional grammar rules and sentence structure are absent. This research addresses this crucial gap by focusing on the detection of product names within unstructured Turkish text. To accomplish this, we propose a deep learning-based NER model which combines a Bidirectional Long Short-Term Memory (BiLSTM) architecture with a Conditional Random Field (CRF) layer, further enhanced by FastText embeddings. To comprehensively evaluate and compare our model’s performance, we explore different embedding approaches, including Word2Vec and Glove, in conjunction with the Bidirectional Long Short-Term Memory and Conditional Random Field (BiLSTM-CRF) model. Furthermore, we conduct comparisons against BERT to assess the efficacy of our approach. Our experimentation utilizes a Turkish e-commerce dataset gathered from the internet, where traditional grammatical and structural rules may not apply. The BiLSTM-CRF model with FastText embeddings achieved an F1 score value of 57.40%, a precision value of 55.78%, and a recall value of 59.12%. These results indicate promising performance in outperforming other baseline techniques. This research contributes to the field of NER by addressing the unique challenges posed by unstructured Turkish text and opens avenues for improved entity recognition in informal language settings, with potential applications across various domains.

Incorporating dynamic attention gating mechanism and pre-trained embedding for Chinese clinical named entity recognition

Clinical Named Entity Recognition (CNER) serves as a cornerstone in clinical research and healthcare informatics, with its primary goal being the discernment and categorization of clinical terminologies present within Electronic Medical Records (EMRs). While deep learning has significantly improved CNER's performance, these models still face challenges in identifying rare or unseen entities. Our study unveils an innovative DAGM-BiLSTM-CRF model that seamlessly integrates data-driven deep learning strategies with human knowledge to surmount this limitation. The distinctive features of this model lie in its integration of a pre-trained character embedding layer, adept at encapsulating nuanced semantic variations at the granularity of individual characters. Furthermore, it incorporates a bidirectional Long Short-Term Memory (BiLSTM) layer, proficient in deriving contextually relevant information by traversing the sequence from both endpoints. The most noteworthy aspect of our model is the inclusion of a Dynamic Attention Gating Mechanism (DAGM) that dynamically recalibrates attention weights based on the traits of the input sequence and a Conditional Random Fields (CRF) layer that considers interdependencies among output labels within a sequence. The evaluation was carried out on the CCKS-2017 CNER benchmark dataset to underscore the superiority of our model over existing cutting-edge methods, thereby validating its proficiency in deciphering the intricacies of Chinese electronic medical records.

Improving Extraction of Chinese Open Relations Using Pre-trained Language Model and Knowledge Enhancement

ABSTRACT Open Relation Extraction (ORE) is a task of extracting semantic relations from a text document. Current ORE systems have significantly improved their efficiency in obtaining Chinese relations, when compared with conventional systems which heavily depend on feature engineering or syntactic parsing. However, the ORE systems do not use robust neural networks such as pre-trained language models to take advantage of large-scale unstructured data effectively. In respons to this issue, a new system entitled Chinese Open Relation Extraction with Knowledge Enhancement (CORE-KE) is presented in this paper. The CORE-KE system employs a pre-trained language model (with the support of a Bidirectional Long Short-Term Memory (BiLSTM) layer and a Masked Conditional Random Field (Masked CRF) layer) on unstructured data in order to improve Chinese open relation extraction. Entity descriptions in Wikidata and additional knowledge (in terms of triple facts) extracted from Chinese ORE datasets are used to fine-tune the pre-trained language model. In addition, syntactic features are further adopted in the training stage of the CORE-KE system for knowledge enhancement. Experimental results of the CORE-KE system on two large-scale datasets of open Chinese entities and relations demonstrate that the CORE-KE system is superior to other ORE systems. The F1-scores of the CORE-KE system on the two datasets have given a relative improvement of 20.1% and 1.3%, when compared with benchmark ORE systems, respectively. The source code is available at https://github.com/cjwen15/CORE-KE.

STIOCS: Active learning-based semi-supervised training framework for IOC extraction

Cyber Threat Intelligence (CTI) contains numerous Indicators of Compromise (IOCs) and contextual information, crucial for understanding threat actors’ behavior and intentions. However, current information extraction predominantly relies on supervised learning algorithms, presenting challenges in the field of CTI for two reasons. Firstly, the scarcity of labeled data with IOCs hampers the effectiveness of supervised learning. Secondly, existing methods struggle to extract comprehensive contextual features, posing difficulties in IOC recognition within CTI. To address these limitations and better suit the unique characteristics of CTI text, this paper introduces STIOCS, a semi-supervised framework that combines active learning and self-training for IOC extraction. STIOCS enhances IOC extraction accuracy and efficiency by leveraging limited labeled data and a rich unannotated corpus. Firstly, the Active Learning (AL) approach uses the Density-based Spatial Clustering of Applications with Noise (DBSCAN) algorithm to select reliable samples that can reduce noise pollution on pseudo-labeling in self-training. The extraction model integrates Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) algorithms to extract local and sequential features from CTI text, respectively. Then, the semantic features are enhanced by using the different sizes of convolutional kernels to fuse the two types of features. Finally, the Conditional Random Fields (CRF) layer is employed to recognize IOC entities. Our experimental results demonstrate the effectiveness and robustness of our proposed method in IOC extraction, even with limited labeled data. Compared to supervised methods, our proposed method is only approximately 40% of the dataset is labeled, the F1 scores are achieved better than the existing methods and exhibit consistent performance improvements as the dataset size increases. STIOCS effectively suppresses weak label noise, reduces training costs, and enhances the recognition model’s performance. It provides a cost-effective training framework for entity extraction in cyber threat intelligence.

ERNIE-based Named Entity Recognition Method for Traffic Accident Cases

Traffic accident case named entity recognition, which helps mine key information in traffic accident texts, plays a vital role in downstream tasks such as the construction of knowledge graphs in road traffic and intelligent policing. In this paper, we construct a named entity recognition model based on the EDE (Entity Data Enhancement)-ERNIE-Bidirectional Gated Recurrent Unit Network (BiGRU)-Conditional Random Field (CRF) to address the current situation of low traffic accident case data and poor recognition of long-text entities. First, the amount of accident case data is enhanced using the entity random substitution method. Next, the text data of traffic accident cases are characterized as a dynamic word vector using the ERNIE pretraining model. Then, the BiGRU network learns the long-distance dependency relationship in the text to enhance the effect of the model on long-text entity recognition. Finally, the result sequence is constrained by the CRF layer to realize the named entity recognition model. The experimental part uses data related to real traffic accident cases in a domestic area. The data enhancement method increases the data volume three times compared to the original data volume. Experimental results show that the EDE-ERNIE-BiGRU-CRF model achieves better F1 values, recall and precision achieved better performance than the entity recognition methods of BERT-BiGRU-CRF, ERNIE-BiGRU-CRF, ERNIE-BiLSTM-CRF, ERNIE-CRF, ERNIE, BiGRU-CRF, ROBERTA-wwm-ext-BiGRU-CRF and verify its effectiveness for entity recognition in traffic accident cases.

Deep Learning-based Sequence Labeling Tools for Nepali

A Part-of-Speech (POS) tagger and Chunker (or shallow parser) are sequence labeling tools, crucial for improving the accuracy of Natural Language Processing (NLP) tasks like parsing, named entity recognition, sentiment analysis, information extraction, and so on. Developing such tools for a low-resource language is an arduous task. Nepali is a relatively resource-poor Indian language and has not been able to evolve from a computational perspective. Therefore, we present effective part-of-speech tagging and chunking tools for the Nepali text using sequential deep learning models—Bidirectional Long Short-Term Memory Network with a Conditional Random Field Layer (BI-LSTM-CRF) and other LSTM-based models exploring both character and word embeddings of the Nepali texts. Word Embedding has been used to capture syntactic as well as semantic information whereas character embedding has been applied to capture the morphological as well as shape information of words and also to handle the out-of-vocabulary problem. The developed chunker is the first statistical chunker for the Nepali language. A baseline model with a Conditional Random Field has also been developed to identify the optimum feature set for the aforementioned tasks. The BI-LSTM-CRF model produced an accuracy of 99.20% and 98.40%, for Nepali POS tagging and chunking, respectively. This is the highest-ever accuracy for Nepali. Thorough error analysis and observations have also been reported with examples. The developed tools can help advance research in Nepali language processing, improve the accuracy of language technology applications, and contribute to the preservation and promotion of the Nepali language.

Named Entity Recognition in Power Marketing Domain Based on Whole Word Masking and Dual Feature Extraction

With the aim of solving the current problems of low utilization of entity features, multiple meanings of a word, and poor recognition of specialized terms in the Chinese power marketing domain named entity recognition (PMDNER), this study proposes a Chinese power marketing named entity recognition method based on whole word masking and joint extraction of dual features. Firstly, word vectorization of the electricity text data is performed using the RoBERTa pre-training model; then, it is fed into the constructed dual feature extraction neural network (DFENN) to acquire the local and global features of text in a parallel manner and fuse them. The output of the RoBERTa layer is used as the auxiliary classification layer, the output of the DFENN layer is used as the master classification layer, and the output of the two layers is dynamically combined through the attention mechanism to weight the outputs of the two layers so as to fuse new features, which are input into the conditional random field (CRF) layer to obtain the most reasonable label sequence. A focal loss function is used in the training process to alleviate the problem of uneven sample distribution. The experimental results show that the method achieved an F1 value of 88.58% on the constructed named entity recognition dataset in the power marketing domain, which is a significant improvement in performance compared with the existing methods.

Part-of-Speech Tagging of Odia Language Using Statistical and Deep Learning Based Approaches

Automatic part-of-speech (POS) tagging is a preprocessing step of many natural language processing tasks, such as named entity recognition, speech processing, information extraction, word sense disambiguation, and machine translation. It has already gained promising results in English and European languages. However, in Indian languages, particularly in the Odia language, it is not yet well explored because of the lack of supporting tools, resources, and morphological richness of the language. Unfortunately, we were unable to locate an open source POS tagger for the Odia language, and only a handful of attempts have been made to develop POS taggers for the Odia language. The main contribution of this research work is to present statistical approaches such as the maximum entropy Markov model and conditional random field (CRF), as well as deep learning based approaches, including the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) to develop the Odia POS tagger. A publicly accessible corpus annotated with the Bureau of Indian Standards (BIS) tagset is used in our work. However, most of the languages around the globe have used the dataset annotated with the Universal Dependencies (UD) tagset. Hence, to maintain uniformity, the Odia dataset should use the same tagset. Thus, following the BIS and UD guidelines, we constructed a mapping from the BIS tagset to the UD tagset. The maximum entropy Markov model, CRF, Bi-LSTM, and CNN models are trained using the Indian Languages Corpora Initiative corpus with the BIS and UD tagsets. We have experimented with various feature sets as input to the statistical models to prepare a baseline system and observed the impact of constructed feature sets. The deep learning based model includes the Bi-LSTM network, the CNN network, the CRF layer, character sequence information, and a pre-trained word vector. Seven different combinations of neural sequence labeling models are implemented, and their performance measures are investigated. It has been observed that the Bi-LSTM model with the character sequence feature and pre-trained word vector achieved a result with 94.58% accuracy.

Identifying business information through deep learning: analyzing the tender documents of an Internet-based logistics bidding platform

PurposeThe tender documents, an essential data source for internet-based logistics tendering platforms, incorporate massive fine-grained data, ranging from information on tenderee, shipping location and shipping items. Automated information extraction in this area is, however, under-researched, making the extraction process a time- and effort-consuming one. For Chinese logistics tender entities, in particular, existing named entity recognition (NER) solutions are mostly unsuitable as they involve domain-specific terminologies and possess different semantic features.Design/methodology/approachTo tackle this problem, a novel lattice long short-term memory (LSTM) model, combining a variant contextual feature representation and a conditional random field (CRF) layer, is proposed in this paper for identifying valuable entities from logistic tender documents. Instead of traditional word embedding, the proposed model uses the pretrained Bidirectional Encoder Representations from Transformers (BERT) model as input to augment the contextual feature representation. Subsequently, with the Lattice-LSTM model, the information of characters and words is effectively utilized to avoid error segmentation.FindingsThe proposed model is then verified by the Chinese logistic tender named entity corpus. Moreover, the results suggest that the proposed model excels in the logistics tender corpus over other mainstream NER models. The proposed model underpins the automatic extraction of logistics tender information, enabling logistic companies to perceive the ever-changing market trends and make far-sighted logistic decisions.Originality/value(1) A practical model for logistic tender NER is proposed in the manuscript. By employing and fine-tuning BERT into the downstream task with a small amount of data, the experiment results show that the model has a better performance than other existing models. This is the first study, to the best of the authors' knowledge, to extract named entities from Chinese logistic tender documents. (2) A real logistic tender corpus for practical use is constructed and a program of the model for online-processing real logistic tender documents is developed in this work. The authors believe that the model will facilitate logistic companies in converting unstructured documents to structured data and further perceive the ever-changing market trends to make far-sighted logistic decisions.

A Study on the Performance of Recurrent Neural Network based Models in Maithili Part of Speech Tagging

This article presents our effort in developing a Maithili Part of Speech (POS) tagger. Substantial effort has been devoted to developing POS taggers in several Indian languages, including Hindi, Bengali, Tamil, Telugu, Kannada, Punjabi, and Marathi; but Maithili did not achieve much attention from the research community. Maithili is one of the official languages of India, with around 50 million native speakers. So, we worked on developing a POS tagger in Maithili. For the development, we use a manually annotated in-house Maithili corpus containing 56,126 tokens. The tagset contains 27 tags. We train a conditional random fields (CRF) classifier to prepare a baseline system that achieves an accuracy of 82.67%. Then, we employ several recurrent neural networks (RNN)-based models, including Long-short Term Memory (LSTM), Gated Recurrent Unit (GRU), LSTM with a CRF layer (LSTM-CRF), and GRU with a CRF layer (GRU-CRF) and perform a comparative study. We also study the effect of both word embedding and character embedding in the task. The highest accuracy of the system is 91.53%.

Fine-grained document-level financial event argument extraction approach

Document-level financial event argument extraction aims to extract a set of structured financial information related to particular financial events from a financial document. This task is challenging because there is complex fine-grained semantic information between financial event arguments, such as cross-document, inner-sentence and context semantic information. Existing approaches are still unable to capture these fine-grained financial semantic features well, leading to the low performance defects. Different from these existing approaches, we propose an end-to-end fine-grained document-level financial event argument extraction approach (FGD-FEAE), which defines event argument extraction as a sequence tagging task. In the encoder, a novel multiple granularity attention layer and a long-short term memory network (LSTM) extension layer are proposed and designed to effectively capture these fine-grained financial semantic information. In the decoder, to avoid the financial semantic confusion problem, a conditional random field (CRF) layer is constructed to jointly tag the financial event arguments. Finally, FGD-FEAE is tested on the two benchmark financial datasets, Chinese financial annotation dataset (ChiFinAnn) and English financial press releases (EFPR), and our own Chinese financial project dataset (ChiFD). These datasets contain a large amount of financial corpus information and complete financial event arguments and provide a solid foundation for verifying the general applicability of the proposed FGD-FEAE. Experimental results show that FGD-FEAE can achieve the better performance than the state-of-the-art approaches, and the performance is improved by 4.69, 5.01 and 3.78 on the three actual financial datasets, respectively.

NGCICM: A Novel Deep Learning-Based Method for Predicting circRNA-miRNA Interactions.

The circRNAs and miRNAs play an important role in the development of human diseases, and they can be widely used as biomarkers of diseases for disease diagnosis. In particular, circRNAs can act as sponge adsorbers for miRNAs and act together in certain diseases. However, the associations between the vast majority of circRNAs and diseases and between miRNAs and diseases remain unclear. Computational-based approaches are urgently needed to discover the unknown interactions between circRNAs and miRNAs. In this paper, we propose a novel deep learning algorithm based on Node2vec and Graph ATtention network (GAT), Conditional Random Field (CRF) layer and Inductive Matrix Completion (IMC) to predict circRNAs and miRNAs interactions (NGCICM). We construct a GAT-based encoder for deep feature learning by fusing the talking-heads attention mechanism and the CRF layer. The IMC-based decoder is also constructed to obtain interaction scores. The Area Under the receiver operating characteristic Curve (AUC) of the NGCICM method is 0.9697, 0.9932 and 0.9980, and the Area Under the Precision-Recall curve (AUPR) is 0.9671, 0.9935 and 0.9981, respectively, using 2-fold, 5-fold and 10-fold Cross-Validation (CV) as the benchmark. The experimental results confirm the effectiveness of the NGCICM algorithm in predicting the interactions between circRNAs and miRNAs.

ANEC: An Amharic Named Entity Corpus and Transformer Based Recognizer

Named Entity Recognition is an information extraction task that serves as a pre-processing step for other natural language processing tasks, such as machine translation, information retrieval, and question answering. Named entity recognition enables the identification of proper names as well as temporal and numeric expressions in an open domain text. For Semitic languages such as Arabic, Amharic, and Hebrew, the named entity recognition task is more challenging due to the heavily inflected structure of these languages. In this study, we annotate a new comparatively large Amharic named entity recognition dataset and make it publicly available. Using this new dataset, we build multiple Amharic named entity recognition systems based on recent deep learning approaches including transfer learning (RoBERTa), and bidirectional long short-term memory coupled with a conditional random fields layer. By applying the Synthetic Minority Over-sampling Technique to mitigate the imbalanced classification problem, our best performing RoBERTa based named entity recognition system achieves an f1-score of 93%, which is the new state-of-the-art result for Amharic named entity recognition.

A novel method for signal labeling and precise location in a variable parameter milling process based on the stacked-BiLSTM-CRF and FLOSS

Unlabeled time series signals collected during manufacturing typically have low value density and must be labeled and intercepted according to the specific application scenario. During variable-parameter milling, particularly high-precision machining, machining parameters vary, and associated discrepancies in vibration signals are small. In this scenario, signal features that are extracted by hand or via deep learning methods cannot typically distinguish machining states via classification models. To solve this problem, a sequence labeling model developed using a stacked bidirectional long short-term memory network with a conditional random field layer (stacked-BiLSTM-CRF) is proposed in this study to automatically label and intercept vibration signals. The stacked BiLSTM receives the shallow features obtained by the short-time Fourier transform of the vibration signals and then outputs the extracted deep features to capture the before and after dependence of the signals. The stacked BiLSTM is then extended by stacking a CRF layer to explicitly model the dependence of signal labels. In a more accurate labeling scenario, the fast low-cost online semantic segmentation algorithm (FLOSS) is used to acquire more fine-grained signal boundary locations after obtaining the frame-level signal label using the stacked BiLSTM-CRF model. In addition, to evaluate model performance, a novel evaluation index for signal labeling is proposed. The feasibility and effectiveness of the proposed method are verified using the vibration signals collected from variable parameter cutting experiments, and results show that the proposed model achieves the best labeling performance of tested methods in nearly all scenarios.

An imConvNet-based deep learning model for Chinese medical named entity recognition

BackgroundWith the development of current medical technology, information management becomes perfect in the medical field. Medical big data analysis is based on a large amount of medical and health data stored in the electronic medical system, such as electronic medical records and medical reports. How to fully exploit the resources of information included in these medical data has always been the subject of research by many scholars. The basis for text mining is named entity recognition (NER), which has its particularities in the medical field, where issues such as inadequate text resources and a large number of professional domain terms continue to face significant challenges in medical NER.MethodsWe improved the convolutional neural network model (imConvNet) to obtain additional text features. Concurrently, we continue to use the classical Bert pre-training model and BiLSTM model for named entity recognition. We use imConvNet model to extract additional word vector features and improve named entity recognition accuracy. The proposed model, named BERT-imConvNet-BiLSTM-CRF, is composed of four layers: BERT embedding layer—getting word embedding vector; imConvNet layer—capturing the context feature of each character; BiLSTM (Bidirectional Long Short-Term Memory) layer—capturing the long-distance dependencies; CRF (Conditional Random Field) layer—labeling characters based on their features and transfer rules.ResultsThe average F1 score on the public medical data set yidu-s4k reached 91.38% when combined with the classical model; when real electronic medical record text in impacted wisdom teeth is used as the experimental object, the model's F1 score is 93.89%. They all show better results than classical models.ConclusionsThe suggested novel model (imConvNet) significantly improves the recognition accuracy of Chinese medical named entities and applies to various medical corpora.

Predicting the potential human lncRNA-miRNA interactions based on graph convolution network with conditional random field.

Long non-coding RNA (lncRNA) and microRNA (miRNA) are two typical types of non-coding RNAs (ncRNAs), their interaction plays an important regulatory role in many biological processes. Exploring the interactions between unknown lncRNA and miRNA can help us better understand the functional expression between lncRNA and miRNA. At present, the interactions between lncRNA and miRNA are mainly obtained through biological experiments, but such experiments are often time-consuming and labor-intensive, it is necessary to design a computational method that can predict the interactions between lncRNA and miRNA. In this paper, we propose a method based on graph convolutional neural (GCN) network and conditional random field (CRF) for predicting human lncRNA-miRNA interactions, named GCNCRF. First, we construct a heterogeneous network using the known interactions of lncRNA and miRNA in the LncRNASNP2 database, the lncRNA/miRNA integration similarity network, and the lncRNA/miRNA feature matrix. Second, the initial embedding of nodes is obtained using a GCN network. A CRF set in the GCN hidden layer can update the obtained preliminary embeddings so that similar nodes have similar embeddings. At the same time, an attention mechanism is added to the CRF layer to reassign weights to nodes to better grasp the feature information of important nodes and ignore some nodes with less influence. Finally, the final embedding is decoded and scored through the decoding layer. Through a 5-fold cross-validation experiment, GCNCRF has an area under the receiver operating characteristic curve value of 0.947 on the main dataset, which has higher prediction accuracy than the other six state-of-the-art methods.

Leveraging Multi-source knowledge for Chinese clinical named entity recognition via relational graph convolutional network.

External knowledge, such as lexicon of words in Chinese and domain knowledge graph (KG) of concepts, has been recently adopted to improve the performance of machine learning methods for named entity recognition (NER) as it can provide additional information beyond context. However, most existing studies only consider knowledge from one source (i.e., either lexicon or knowledge graph) in different ways and consider lexicon words or KG concepts independently with their boundaries. In this paper, we focus on leveraging multi-source knowledge in a unified manner where lexicon words or KG concepts are well combined with their boundaries for Chinese Clinical NER (CNER). We propose a novel method based on relational graph convolutional network (RGCN), called MKRGCN, to utilize multi-source knowledge in a unified manner for CNER. For any sentence, a relational graph based on words or concepts in each knowledge source is constructed, where lexicon words or KG concepts appearing in the sentence are linked to the containing tokens with the boundary information of the lexicon words or KG concepts. RGCN is used to model all relational graphs constructed from multi-source knowledge, and the representations of tokens from multi-source knowledge are integrated into the context representations of tokens via an attention mechanism. Based on the knowledge-enhanced representations of tokens, we deploy a conditional random field (CRF) layer for named entity label prediction. In this study, a lexicon of words and a medical knowledge graph are used as knowledge sources for Chinese CNER. Our proposed method achieves the best performance on CCKS2017 and CCKS2018 in Chinese with F1-scores of 91.88% and 89.91%, respectively, significantly outperforming existing methods. The extended experiments on NCBI-Disease and BC2GM in English also prove the effectiveness of our method when only considering one knowledge source via RGCN. The MKRGCN model can integrate knowledge from the external lexicon and knowledge graph effectively for Chinese CNER and has the potential to be applied to English NER.

Negation and uncertainty detection in clinical texts written in Spanish: a deep learning-based approach.

Detecting negation and uncertainty is crucial for medical text mining applications; otherwise, extracted information can be incorrectly identified as real or factual events. Although several approaches have been proposed to detect negation and uncertainty in clinical texts, most efforts have focused on the English language. Most proposals developed for Spanish have focused mainly on negation detection and do not deal with uncertainty. In this paper, we propose a deep learning-based approach for both negation and uncertainty detection in clinical texts written in Spanish. The proposed approach explores two deep learning methods to achieve this goal: (i) Bidirectional Long-Short Term Memory with a Conditional Random Field layer (BiLSTM-CRF) and (ii) Bidirectional Encoder Representation for Transformers (BERT). The approach was evaluated using NUBES and IULA, two public corpora for the Spanish language. The results obtained showed an F-score of 92% and 80% in the scope recognition task for negation and uncertainty, respectively. We also present the results of a validation process conducted using a real-life annotated dataset from clinical notes belonging to cancer patients. The proposed approach shows the feasibility of deep learning-based methods to detect negation and uncertainty in Spanish clinical texts. Experiments also highlighted that this approach improves performance in the scope recognition task compared to other proposals in the biomedical domain.